Testing is an important, but time-consuming, part of software development. Testing processes commonly aim to execute a significant portion of the code of which a software application is composed, in order to determine whether the application operates in the manner in which it was designed to operate. The degree to which an application or computer system is tested is known as the “coverage” of the test. A high coverage indicates that the application has been thoroughly tested. However, most applications are so large and complicated that, in practice, it is impossible for a test to achieve complete coverage. For example, the existence of multiple different execution paths within an application means that it is practically impossible to test each and every line of code, and it is often too time-consuming to achieve a high coverage.
Various tools exist to automate the testing of applications and computer systems. For example, IBM Corporation produces a suite of test tools including IBM® Rational® Performance Test Server and IBM® Rational® Test Workbench, which aim to reduce the time taken to test complex applications by automating much of the testing process.
Automated test tools nevertheless require a significant amount of human involvement. For example, consider an application that is designed to process a message received via a communications network, in which the message contains several attributes or fields, each of which can have a number of possible values. The testing of such an application typically requires a test engineer to construct a large number of test messages containing values that are intended to cause every aspect of the application's functionality to be tested. Constructing such test messages is very time-consuming, and it commonly takes many weeks or months to construct a set of test messages for a typical application. However, even when many person-hours are taken to construct a set of test messages, it has been found that the set of test messages do not achieve a high coverage and/or contain invalid values for attributes in the messages. The presence of invalid values is particularly troublesome, since it may cause “false positive” errors to be detected (e.g. where a correctly-functioning program wrongly appears to contain errors because it has been given invalid data to process) or cause “true positive” errors to be missed (e.g. where an incorrectly-functioning program appears to contain no errors because the invalid data masks the presence of errors).
Thus, there is a need for an improved method for generating data for use in testing applications and computer systems.